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Abstract 


A  method  for  extracting  the  motion  parameter*  of  aeveral  independently  moving 
object*  from  displacement  field  information  is  described.  The  method  is  based  on  a 
generalized  Hough  transform  technique.  Some  of  the  problems  of  this  technique  are 
addressed  and  appropriate  solutions  are  proposed.  A  modified  multipass  Hough  transform 
approach  has  bom  implemented,  where  in  each  pass  windows  are  located  around  objects 
and  the  transform  is  applied  only  to  the  displacement  vectors  contained  in  these  windows. 
The  windows  are  determined  by  the  degree  to  which  the  displacement  field  is  locally 
inconsistent  with  previously  found  motion  transformations.  Thus,  die  sensitivity  of  the 
Hough  transform  to  local  events  is  increased  and  the  motion  parameters  of  small  objects 
can  be  detected  even  in  a  noisy  displacement  field,  -w  also  use»a  multi-resolution  scheme 
in  both  the  image  plane  and  the  parameter  space  and  thus  reduce>.the  computational  cost 
of  the  technique.  The  method  is  demonstrated  by  experiments  based  on  artificial  images 
with  four  parameters  of  2-D  motion:  rotation,  expansion  and  translation  in  both  axes. 


This  paper  will  appear  in  the  proceedings  of  the  DARPA  Image  Understanding  Workshop, 
to  be  hdd  in  Arlington  VA,  June  1983. 
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1.  Introduction 


A  time-varying  scene  may  contain  several  independently  moving  objects  with  unknown 
location,  shape  and  3-D  structure.  The  interpretation  of  such  a  scene  includes  the 
computation  of  the  motion  parameters  of  the  camera  and  each  moving  object.  This 
information  is  useful  in  areas  such  as  robotics  and  navigation.  It  could  also  be  used  as 
an  intermediate  stage  for  achieving  the  tasks  of  object-surround  separation  and  structure 
determination. 

Our  approach  for  recovering  the  motion  parameters  is  based  on  two  phases.  First, 
we  compute  a  displacement  field,  composed  of  vectors  describing  the  displacement  of  image 
elements  from  one  image  to  the  next  (see  section  2).  In  this  paper  we  assume  a  dense 
displacement  field,  but  the  second  phase  is  basically  independent  of  this  assumption.  Each 
displacement  vector  is  assigned  a  weight  representing  its  reliability. 

In  the  second  phase  the  displacement  field  is  interpreted  and  the  motion  parameters 
are  recovered.  This  phase,  which  is  the  main  concern  of  the  paper,  is  based  on  the 
generalized  Hough  transform  technique  [BAL81a].  In  this  technique  the  motion  parameters 
are  represented  by  a  discrete  multi-dimensional  parameter  space  where  each  dimension 
corresponds  to  one  of  the  parameters.  Each  point  in  this  space  uniquely  characterizes  a 
motion  transformation,  defined  by  the  corresponding  parameter  values.  A  displacement 
vector  "votes”  for  a  point  in  the  space  if  the  corresponding  transformation  is  consistent  with 
this  vector.  The  points  receiving  the  most  votes  are  likely  to  represent  the  motion 
parameters  of  different  objects. 


There  are  a  few  techniques  described  in  the  literature  which  use  the  Hough  transform 
for  dealing  with  scenes  containing  several  moving  objects.  Fennema  and  Thompson  [FEN79] 
compute  spatial  and  temporal  gradients  of  the  image.  A  Hough  transform  technique  is  used 
to  detect  velocities  which  are  consistent  with  a  significant  portion  of  the  gradient  field.  A 
multipass  approach  is  used:  first  the  most  prominent  peak  in  the  Hough  transform  is 
found  and  thus  the  velocity  of  the  largest  object  is  recovered.  Then  the  image  points  which 
are  consistent  with  this  velocity  are  removed  and  a  new  peak  is  looked  for.  The  process  is 
repeated  until  no  further  objects  are  found.  This  system  is  restricted  to  translation.  It  also 
has  problems  in  recognizing  significant  peaks  [TH081]. 

Ballard  and  Kimball  [BAL81b]  consider  the  case  of  general  3-D  motion  of  rigid 
objects,  but  assume  knowledge  of  depth  information.  A  Hough  transform  technique  for 
computing  the  motion  parameters  from  3-D  optic  flow  is  implemented.  The  simulation,  as 
described  in  their  report,  assumes  only  one  moving  object,  but  it  is  argued  that  a  multipass 
approach  would  handle  the  case  of  several  moving  objects. 

Jayaramurthy  and  Jain  [JAY82]  describe  an  implementation  of  the  Hough  transform 
technique  for  computing  motion  parameters  directly  from  the  intensity  information.  Several 
moving  objects  are  allowed,  but  a  stationary  background  and  translational  motion  are 
assumed. 

One  of  the  well  known  advantages  of  the  Hough  technique  is  its  relative  insensitivity 
to  noise  and  partially  incorrect  or  occluded  data.  Another  advantage  is  its  ability  to  detect 
consistency  in  the  image.  In  our  case  it  can  group  together  displacement  vectors  which 
satisfy  the  same  motion  parameters  and  presumably  belong  to  one  object. 
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On  the  other  hand,  the  Hough  technique  has  a  few  disadvantages.  It  is  insensitive 
to  spatial  relations  in  the  displacement  field.  Thus,  a  group  of  non-ad jacent  elements,  which 
incidently  vote  for  the  same  motion  transformation,  may  be  considered  as  representing  one 
object,  whereas  the  motion  parameters  of  a  small  object  may  be  difficult  to  detect.  The 
technique  also  has  high  computational  cost.  Fine  resolution  in  the  parameter  space,  which  is 
related  to  the  accuracy  of  the  final  results,  requires  large  amounts  of  memory  and 
computation  time. 

This  paper  addresses  these  problems.  A  few  ideas  are  examined  in  a  restricted  case  of 
2-D  motion  with  four  parameters  (rotation,  expansion  and  translation  in  both  axes).  An 
analysis  of  reliability  and  efficiency  considerations  is  presented  (section  32)  and  new 
solutions  are  proposed  (section  4).  A  modified  multipass  Hough  transform  approach  has 
been  implemented,  where  in  each  pass  windows  are  located  around  objects  and  the 
transform  is  applied  only  to  the  displacement  vectors  contained  in  these  windows.  The 
windows  are  determined  by  the  degree  to  which  the  displacement  field  is  locally  inconsistent 
with  previously  found  motion  transformations.  Thus,  the  sensitivity  of  the  Hough  transform 
to  local  events  is  increased  and  the  motion  parameters  of  small  objects  can  be  detected 
even  in  a  noisy  displacement  field.  We  also  use  a  multi-resolution  scheme  in  both  the 
image  plane  and  the  parameter  space  and  thus  reduce  the  computational  cost  of  the 
technique.  These  ideas  are  demonstrated  by  experiments  based  on  artificial  images  (section 

5). 
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2.  Computing  i  Displacement  Field  and  a  Weight  Plane 

In  the  first  phase  of  the  algorithm  we  compute  a  displacement  Geld  from  two 
sampled  images.  These  images  contain  several  objects  which  are  moving  independently.  The 
background  is  considered  as  one  of  the  objects.  The  motion  of  each  object  is  composed  of 
rotation,  expansion  and  translation.  It  can  be  represented  by  the  following  affine 
transformation: 

(2.1)  i'  =  (l+expanXcos(rot)i-sin(rot)  j)+trj 

(22)  j'  =  (l+expanXsm(rot)  i+cos(rot)  jj+tr2 

where  (i j)  is  a  pixel  in  the  first  image,  (i',j')  is  the  corresponding  pixel  in  the  second 
image  and  rot,  expan,  trj  and  tr2  are  the  motion  parameter  values. 

The  displacement  field  can  be  described  by  {(Dj(i,j),  D^ij))}  where  (Dj(i,j),  Etyij)) 
represents  the  displacement  vector  at  the  (i,j)  pixel  in  the  first  image.  We  compute  it  by 
using  the  Horn  and  Schunck  technique  [HOR80]  (however,  the  second  phase  of  our 
algorithm  is  almost  independent  of  this  specific  choice).  In  order  to  use  this  tehnique  we 
assume  a  small  displacement  at  each  pixel  and  absence  of  illumination  effects.  It  starts  by 
calculating,  at  each  pixel,  the  spatial  gradient  (Ej,  E2)  and  the  temporal  derivative  Eg. 
The  assumption  that  the  brightness  of  a  particular  point  in  the  scene  is  constant  over  time 
provides  the  following  constraint: 

(23)  E1D1+E2D2+Et  =  0 

The  assumption  of  the  smoothness  of  the  displacement  field  provides  another  constraint. 

An  error  function  can  represent,  for  a  given  displacement  field,  the  degree  of 
departure  from  these  constraints.  The  technique  is  based  on  iteratively  minimizing  this 
function.  Ideally,  the  resulting  field  (Dj,  P2)  should  satisfy  the  following  equations,  derived 
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from  equations  (2.1)  and  (22): 

(2.4)  i+Dj(ij)  =  (l+expanXcos(rot)  i-sin(rot)  jj+trj 
(25)  j+D2(»,j)  =  (l+expanXan(rot)  i+cos(rot)  fl+ti2 

where  roc,  expan,  trj  and  tr2  are  the  motion  parameter  values  in  the  (i,j)  pixel. 

Figure  1  shows  two  pairs  of  artificial  images  which  contain  several  independently 
moving  objects.  The  motion  parameters  of  each  object  are  specified  in  tables  S.l  and  52. 
Figure  2  shows  the  result  of  applying  the  Horn  and  Schunck  technique  to  these  images. 

The  smoothness  constraint  is  violated  at  the  boundaries  of  independently  moving 
objects.  Therefore,  the  computed  displacement  values  in  these  areas  are  incorrect. 

Fortunately,  these  areas  can  be  detected  by  using  the  error  function  which  represents  the 

departure  from  the  constraints.  High  values  of  the  error  function  indicate  that  the 
constraints  are  not  satisfied  and  the  computed  displacement  values  are  unreliable. 

For  each  displacement  vector  we  compute  an  associated  weight  such  that  high 
reliability  (low  value  of  the  error  function)  is  represented  by  a  value  close  to  1  and  low 
reliability  by  a  value  close  to  0.  An  appropriate  relation  between  the  error  function, 
erf(i,j),  and  the  weight,  W(i,j),  can  be  obtained  by  the  function 

(2.6)  W(ij)  =  e-^jyk 

The  parameter  k  was  experimentally  determined  as  0.07.  However,  this  value  need  to  be 
decreased  with  noisier  data.  Figure  3  shows  the  weight  planes  computed  for  the 

displacement  fields  in  figure  2.  When  the  Hough  transform  is  computed  later  the 

influence  ('voting'  power)  of  each  displacement  vector  will  be  proportional  to  its  associated 
weight. 


Figure  la:  Intensity  images  used  in  the  first  experiment  (the  white  lines  only 


emphasize  the  contours  of  the  objects  and  are  not  part  of  the  images); 


-  object  A  is  the  background, 


-  object  B  is  the  large  circle  in  the  center  of  the  image, 


-  object  C  is  the  small  circle  in  the  upper  left  comer. 


Figure  l.b:  Intensity  images  used  in  the  second  experiment; 

-  object  A  is  the  background, 

-  object  B  is  the  circle  in  the  upper  right  corner, 

-  object  C  >  tire  circle  •  hich  partially  occludes  object  B, 

-  object  D  i  Hr  in  the  left  part  of  the  image, 

-  object  E  is  -  small  circle  in  the  lower  part. 
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Fl*n«  2:  Samples  of  the  displacement  fields,  (a)  First  experiment, 
(b)  Second  experiment. 
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3.  The  Generalized  Hough  Transform  Technique 
3.1  General  Description 

The  motion  parameters  can  be  represented  by  a  4-dimensional  parameter  space  where 
each  dimension  corresponds  to  one  of  the  motion  parameters:  rotation  (rot),  expansion  or 
contraction  (expan),  vertical  translation  (trj)  and  horizontal  translation  (tr2).  Each  point  in 
this  space  uniquely  characterizes  a  motion  transformation  in  the  image. 

We  say  that  a  displacement  vector  (Dj(i,j),  Etyij))  is  consistent  with  a  point 
(rot, expan.tr )  in  the  parameter  space  if  it  satisfies  equations  (2.4)  and  (2.5).  Let  us 
define  a  subset  B(i,j)  of  the  parameter  space  as  the  set  of  all  the  points  in  this  space 
which  are  consistent  with  (Dj(ij),  D^ij)).  Using  the  definition  in  [BAL81a],  the  Hough 
transform  is  the  following  function,  defined  on  the  parameter  space: 

(3.1)  H(rot  .expan  ,trj.tr2)  =  2  W(ij) 

(rot^xpan,trj,tr2)  €  B(i,j) 

i.e.,  H(rot, expan ,trj,tr2)  is  the  sum  of  the  associated  weights  of  all  the  displacement  vectors 
which  are  consistent  with  the  point  (rot  .expan, ft]  ,tr2).  High  values  of  the  Hough  transform 
represent  motion  transformations  which  are  consistent  with  a  significant  portion  of  the 
vectors.  The  use  of  the  weight  function  W  should  prevent  unreliable  values  of  displacement 
vectors  from  creating  false  peaks. 

In  practice,  we  assume  a  limited  velocity  of  objects,  i.e.  minimal  and  maximal  values 
for  each  parameter.  The  corresponding  intervals  are  quantized  and  thus  each  parameter  is 
represented  by  a  discrete  set  of  values.  The  parameter  space  is  the  cartesian  product  of 


these  sets. 
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For  each  displacement  vector  (Dj(i,j),  E^ij))  and  each  pair  (rot  .expan)  of  rotation 
and  expansion,  there  exists  exactly  one  pair  of  translations  which  satisfies 

equations  (2.4)  and  (25).  If  trj*  and  tr2*  are  within  the  limits  of  the  respective 
dimensions  of  the  parameter  space,  then  we  can  find  exactly  one  pair  (trj, trj)  such  that  trj 
and  tr2  are  sampled  values  and 

(3 2)  trj-res/2  s  trj*  <  trj+res/2 

(33)  trj-res/2  ss  tr2*  <  tr2+res/2 

where  res  is  the  resolution  of  the  translation  variables  in  the  parameter  space.  In  this  case 
we  say  that  (Dj(ij),  DjGj))  'votes'  for  (rot,expan,trj,tr2),  i.e.,  it  contributes  its  weight  to 
H(rot^xpan,trj,tr2). 

Finally,  among  the  points  whose  Hough  transform  exceeds  a  given  threshold  in  the 
parameter  space,  local  maxima  are  found.  These  represent  the  hypothetical  motions  of 
objects  in  the  image. 

32  Reliability  and  Efficiency  Considerations 

The  resolution  of  the  parameter  space  should  be  determined  by  a  few  considerations: 
the  signal  to  noise  ratio  (SNR),  the  required  accuracy,  the  computation  time  and  the 
required  storage  space. 

For  each  independently  moving  object  in  the  scene,  the  SNR  in  the  parameter  space 
can  be  defined  as: 

,,4v  SNR  =  no.  of  votes  for  the  object  morion 

average  no.  of  votes  for  each  parameter  value 

(for  a  different  definition  see  [BR082]).  If  the  SNR  b  low,  then  false  peaks,  higher  than 
the  value  corresponding  to  the  object,  can  be  created.  Thus  the  detection  of  the  object's 
motion,  by  a  straightforward  Hough  technique,  may  be  difficult  or  impossible. 
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Let  us  assume  that  the  multiplicative  parameters  of  rotation  and  expansion  are 
quantized  into  pj  elements  each  and  that  the  translation  parameters  are  quantized  into  P2 
elements  each.  Then  the  parameter  space  includes  pj^p elements.  Let  n  be  the  number  of 
displacement  vectors  which  are  considered  in  the  computation  of  the  Hough  transform. 
According  to  the  voting  process  described  in  section  (3.1),  for  each  displacement  vector  and 
each  pair  (rot,expan),  there  exits  at  most  one  pair  (tr^tr^  of  translations  such  that  the 
displacement  vector  votes  for  (rot.expan.trj.ti^)-  Assuming  that  trj  and  are  likely  to  be 
within  the  limits  of  the  respective  dimensions  (and  that  is  the  case  in  our  experiments)  we 
can  estimate  the  average  number  of  votes  for  each  parameter  value  as  npj^pj^p^)  = 
n/p2^.  If  c  represents  the  fraction  of  the  image  covered  by  an  object,  then  it  contains  cn 
displacement  vectors,  where  0<csl.  Thus,  we  can  estimate  the  SNR  by: 

(3  S)  SNR  -  f  2  =  cp22 
n/p2Z 

If  for  reliable  detection  of  the  object,  the  SNR  should  be  larger  than  some  threshold 
t,  then  P2  should  satisfy  the  constraint  P2  &  Vt/c.  For  example,  if  t=10  and  c=0.01  then 
P2  should  be  at  least  32.  This  observation  also  indicates  that  for  a  given  P2>  the  motion 
transformation  of  a  small  object  may  be  difficult  to  detect. 

The  second  consideration  is  the  required  accuracy.  The  parameter  resolution  can  be 
dynamically  modified  to  fit  the  expected  constraints  of  the  task  domain.  If,  for  example, 
the  maximal  value  of  rotation  is  1/8  radian,  the  minimal  value  is  -1/8  radian  and  the 
required  resolution  is  1/128  radian,  then  pj  should  be  at  least  32. 
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The  third  consideration  is  computation  time.  Computationally,  the  most  expensive 
process  is  the  voting  process.  We  saw  in  section  (3.1)  that  the  basic  operation  in  this 
process  is  computing  trj,  for  each  displacement  vector  and  each  pair  (rot, expan). 
Therefore,  the  voting  process  takes  approximately  snpj2  time  units,  where  each  basic 
operation  takes  s  time  units. 

The  fourth  consideration  is  the  required  memory  for  the  parameter  space  which 
includes  pj^2  dements.  If  we  combine  the  requirements  for  high  SNR  and  high  accuracy 
we  may  have 

(3.6)  pj^2  a:  324  >  1000000 

In  such  a  large  array,  finding  local  maxima  also  becomes  a  computationally  expensive  task. 

Finally,  assuming  that  we  want  to  obtain  a  given  accuracy  and  we  are  given  a 
storage  space  with  a  fixed  size,  the  optimal  values  of  pj  and  p2  can  be  determined.  Let  us 
suppose  that  the  image  contains  m2  pixels  and  that  the  origin  of  the  coordinate  system  is 
in  the  center  of  the  image.  Then,  using  a  resolution  of  cj  in  the  multiplicative  parameters 
of  rotation  and  expansion  can  cause,  at  the  boundary  of  the  image,  a  displacement  error 
of  mcj/2.  Therefore,  it  is  reasonable  to  quantize  the  parameter  space  in  such  a  way  that 
me]  =*  e2,  where  e2  is  the  resolution  of  translation.  Consequently,  if  m  is  multiplied  by  4, 
for  example,  then  pj  should  be  multiplied  by  2  and  P2  should  be  divided  by  2. 


d-  -  '  .rifci  '  •  - 
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4.  Computing  Motion  Parameters  from  Displacement  field  Information 
4.1  Key  Ideas 

The  proposed  method  is  intended  to  reduce  the  problems  mentioned  in  the  last 
section  and  to  test  mechanisms  for  solving  such  problems  for  even  more  difficult  tasks,  e  g. 
recovering  the  motion  parameters  of  3-D  motion  with  ax  degrees  of  freedom. 

The  key  ideas  which  are  used  for  accomplishing  this  goal  are  the  following: 

1)  Given  a  large  displacement  field  (such  as  the  128x128  array  in  the  experiments), 
we  can  compute  the  motion  parameters  of  large  objects  by  using  a  coarse  resolution  field. 
Such  a  field  can  be  obtained  by  uniformly  sampling  the  initial  field.  In  this  way,  we  can 
considerably  reduce  the  computation  time. 

2)  We  can  find  the  motion  parameters  of  a  given  small  object  by  locating  a  window 
around  the  object  and  applying  the  Hough  transform  only  to  the  displacement  vectors 
contained  in  this  window.  Such  a  window  can  be  located  by  using  a  multipass  approach 
(see  next  section).  By  focusing  our  attention  to  die  window,  we  increase  the  proportion  of 
the  vectors  contained  in  the  object,  i.e.,  we  increase  c  in  equation  (35).  We  can  now 
decrease  P2  and  still  find  the  motion  parameters  of  the  object.  This  technique  enables  us  to 
detect  small  objects  and  save  time  and  storage  space. 

3)  Even  with  a  limited  memory  size,  we  can  find  accurate  parameter  values  by 
iteratively  using  the  Hough  technique.  In  each  iteration  we  quantize  the  parameter  space 
around  the  values  estimated  in  the  previous  iteration,  using  a  finer  resolution.  Other 
methods  for  reducing  the  required  space  in  Hough  techniques  can  be  found  in  [0R061, 
SL081J. 
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42  Description 
42.1  General 

The  algorithm  is  based  on  repeatedly  executing  a  basic  cycle  of  operations.  The  input 
to  each  cycle  includes: 

1)  A  list  L  of  motion  transformations  which  were  computed  in  previous  cycles 
(initially  L  is  an  empty  list). 

2)  A  binary  mask  array  A  in  registration  with  the  displacement  field.  Each  element 
in  A  is  either  1  or  0:  1  if  the  corresponding  displacement  vector  is  consistent  with  one  of 
the  already  computed  motion  transformations;  0  otherwise.  Initially  all  the  entries  in  this 
array  are  set  to  0. 

Each  cycle  is  composed  of  the  following  steps: 

1)  Locate  windows  in  the  image  which  contain  relatively  dense  clusters  of  0-entries  in 
A.  Initially  there  is  one  window  consisting  of  the  whole  image. 

2)  For  each  window  compute  the  Hough  transform  and  hypothesize  (see  section  4 23) 
the  motion  transformations. 

3)  Test,  sequentially,  the  hypothesized  transformations.  The  test  is  done  by 
considering  the  0-entries  in  A  that  are  contained  in  the  window,  and  summing  the  weights 
of  the  associated  displacement  vectors  which  are  consistent  with  the  hypothesized 
transformation.  If  the  sum  is  higher  than  a  given  threshold,  the  transformation  b 
confirmed.  In  thb  case  it  b  added  to  the  list  L  and  the  array  A  b  updated 
correspondingly. 


Ml  Locating  Windows 


A  window  can  be  described  as  a  set  {(i,j):  iQ^i<ij,  jj^jcjj}.  For  implementation 
reasons,  we  consider  only  windows  such  that 

•0*  '1*  ft*  h  €  {0,43,  .,128} 
and 

•r*0»  il-JD  €  {8,16,32,64} 

For  each  window,  we  define  a  criterion  function  CR  by: 

,4  jv  ££  =  no.  of  Gentries  of  A  in  the  window 

'  ‘  ’  Varea  of  the  window 

We  look  for  windows  with  high  values  of  CR.  Such  windows  contain  dense  dusters  of 
0-entries  in  A.  The  use  of  square  root  in  the  denominator  of  equation  (4.1)  means  that 
this  density  can  be  lower  as  the  window  becomes  larger.  If  we  would  eliminate  the  square 
root  in  this  equation,  then  too  small  windows,  which  contain  only  0-entries,  would  be 
chosen.  If  we  do  not  find  any  appropriate  windows,  i.e.  windows  whose  criterion  function 
exceeds  a  given  threshold,  the  process  is  stopped.  The  reason  is  that  probably  there  are  no 
more  objects  whose  motion  transformation  has  not  already  been  found. 

Figure  4  shows  the  windows  found  in  the  second  cyde  when  the  method  is  applied 
to  the  images  in  figure  1.  Figure  5  shows  the  A  arrays  when  the  processes  are  stopped. 
The  black  areas  in  figure  S,  which  represent  the  0-entries  in  the  A  arrays,  correspond  to 
incorrect  values  of  displacement  vectors  in  the  boundaries  of  the  objects. 


r 


Wigan  4:  Optimal  windows  found  in  the  A  arrays  during  the  second  cycle  of 
the  experiments,  (a)  First  experiment,  (b)  Second  experiment. 


Figure  S:  Final  A  arrays,  (a)  First  experiment,  (b)  Second  Experiment. 
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4 23  Tbe  Hypothesizing  Phase 

Before  we  start  the  voting  process  of  the  displacement  vectors  in  a  given  window,  we 
have  to  decide  which  vectors  take  part  in  this  process  and  how  the  parameter  space  is 
defined.  If  the  window  contains  no  more  than  1024  elements,  then  all  of  them  take  part 
in  the  voting  process.  Otherwise,  for  efficiency  considerations  (see  section  4.1),  we  will 
utilize  a  uniformly  sampled  subset  of  1024  elements.  For  example,  if  the  window  is  32x64 
elements,  we  define  a  sub-array  of  32x32  elements  by  choosing  all  the  elements  (i,j)  such 
that  j  is  even. 

The  parameter  space  is  an  adjustable  4-D  array  ('adjustable'  means  that  the  number 
of  elements  in  each  dimension  is  not  fixed)  which  contains  17*  (=>90000)  elements.  We 
assume  that  the  rotation  is  limited  to  0.125  radians  in  each  direction,  the  expansion  (or 
contraction)  is  limited  to  0.125  and  the  translation  is  limited  to  8  pixels  in  each  direction. 
The  axes  which  correspond  to  rotation  and  expansion  contain  pj  elements  each  and  the 
axes  which  correspond  to  the  translations  contain  P2  elements  each.  If  the  length  of  the 
window  is  at  least  64  elements  then,  according  to  the  argument  described  at  the  end  of 
section  32  for  equalizing  the  effective  parameter  resolutions,  we  choose  Pi=P2=17*  otherwise 
pj  is  decreased  and  P2  is  increased.  So,  for  example,  if  the  window  is  16x16,  we  choose 
Pj=9  and  P2=31. 

After  the  voting  process  is  finished,  local  maxima  in  the  Hough  transform  are 
determined.  From  these  candidates,  the  ones  that  exceed  a  given  threshold  are  selected. 
The  threshold  is  proportional  to  the  number  of  all  the  voting  displacement  vectors.  If  the 
resolution  of  the  translation  axes  is  more  than  1/2  pixel,  we  define  a  new  parameter  apace, 
around  each  maxima  point,  with  finer  resolution.  We  then  recompute  tbe  Hough 
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transform.  The  process  is  repeated  until  we  achieve  a  resolution  of  1/2  pixel  at  most.  At 
the  end  of  this  process  we  have  a  set  of  hypothesized  transformations. 

42.4  The  Testing  Phase 

In  this  phase  we  sequentially  test  the  hypothesized  transformations  in  the  order  of 
their  Hough  transform  values  in  the  parameter  space.  We  test  only  the  transformations 
which  are  still  not  included  in  the  list  L  of  confirmed  transformations.  When  we  test  a 
given  transformation,  we  check  all  the  displacement  vectors  with  associated  0-entry  in  the 
corresponding  window.  We  sum  the  weights  of  such  vectors  which  are  consistent  with  the 
hypothesized  transformation.  We  also  compute  a  threshold  proportional  to  the  number  of 
0-entries  in  A,  contained  in  the  window.  If  the  sum  is  higher  than  the  threshold,  we 
accept  the  transformation,  add  it  to  the  list  L  and  update  the  array  A.  In  the  current 
implementation,  the  process  is  stopped  if  we  do  not  accept  any  transformation  in  any  of 
the  windows. 


5.  Experiments 


We  performed  two  experiments  based  on  two  pain  of  128x128  artificial  images 
(figure  1).  In  the  experiments  the  objects  were  transformed  according  to  the  upper  values 
in  each  entry  in  tables  S.l  and  52.  The  lower  numbers  in  these  entries  are  the  computed 
parameten. 
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1 

1 
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1 

1 

rotation  1 
(radians)  1 

1 

expansion  I 

1 

1 

vertical  1 
translation  1 
(pixels)  1 
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translation  1 
(pixels)  1 
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0.  1 
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1  computed 
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0.  1 
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1 
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1 

0.  1 

i 
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0.  1 

1  computed 
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0.0781  ( 

0.  1 

0.  1 

0.  1 
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1 

l 

1 

0.  1 

1 

0.  1 

1 

0.  1 

2.  1 

1  computed 

1 

0.  1 

0.  1 

0.  1 

2.  1 

table  5.1  •  first  experiment 


In  the  first  stage  -  the  displacement  field  was  computed  (figure  2).  Note  die  errors 
at  the  boundaries  of  the  objects  which  correspond  to  low  values  in  the  weight  planes 
(figure  3). 

In  experiment  1,  during  the  first  cycle  of  the  algorithm  for  computing  the  motion 
parameten,  the  motion  transformations  of  objects  A  and  B  were  detected  (see  the  results  in 
table  5.1).  In  the  second  cycle,  taro  windows  were  located  around  areas  in  the  mask  array 
A  with  relatively  dense  dusten  of  (Gentries  (figure  4),  but  only  the  window  around  object 
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C  gave  a  positive  result  -  the  motion  transformation  of  object  C.  In  the  final  cycle  no 
appropriate  windows  were  found  in  the  array  A  (figure  5). 


Corresponding  results  from  experiment  2  are  also  shown  in  figure  4  and  figure  S.  The 
computed  transformations  are  shown  in  table  52. 
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1  computed 
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0.12  1 

1 

-0.1  1 
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0.125  l 

-0.0937  1 

-02 

0.  1 

1  object  E 
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1 

l 

1 

0.  1 

1 

0.125  1 

- 1.1 

0.7  1 

1  computed 

l 

0.  1 

0.0625  (•)  1 

-1.05 

0.6  1 

table  52  -  second  experiment 

(*)  The  large  error  indicated  in  this  entry  is  due  to  the  small  size  of  object  E  (radius  =  8 
pixels)  which  reduces  the  possible  resolution  in  the  measurements  of  rotation  and  expansion. 
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6.  Conclusions  and  Extensions 

This  work  demonstrates  an  efficient  and  robust  algorithm,  based  on  the  Hough 
technique,  for  recovering  motion  parameters  in  scenes  containing  several  independently 
moving  objects.  An  hierarchical  approach,  combined  with  a  windowing  scheme,  is 
implemented  in  order  to  deal  with  objects  of  different  size.  The  storage  space  and 
computation  time  can  be  limited,  while  still  computing  the  motion  parameters  very 
accurately  and  distinguishing  between  real  objects  and  noise  effects. 

We  hope  to  extend  this  work  for  sequences  of  images  and  for  recovering  the  3-D 
motion  parameters  of  rigid  objects.  However,  the  latter  task  is  much  more  difficult  than 
recovering  2-D  motion  parameters.  In  the  2-D  case  each  vector  contributes  two  constraints 
(equations  2.4  and  2.5)  whereas  in  the  3-D  case,  assuming  that  depth  information  is 
unknown,  each  vector  contributes  only  one  constraint.  Therefore,  the  signal  to  noise  ratio 
in  the  parameter  space  (section  32)  is  much  lower.  In  addition,  we  expect  to  have 
problems  of  ambiguity  in  the  interpretation  of  noisy  displacement  fields,  where  a  group  of 
motion  transformations  can  be  equally  consistent  with  the  data.  In  such  cases  a 
probabilistic  approach  might  be  more  suitable.  We  also  plan  to  implement  less  restricted 
methods  for  computing  a  displacement  field  or  other  equivalent  information,  to  use  the 
motion  information  for  object-surround  separation,  and  to  test  the  method  in  real  scenes. 
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